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(54) Apparatus and method for synchronizing video and audio mpeg streams in a video playback 
device 



(57) There is disclosed an MPEG decoder compris- 
ing: 1) a packetized elementary stream (PES) interface 
for receiving a plurality of packetized elementary 
streams associated with a single video program; 2) a 
presentation time stamp (PTS) detection circuit for de- 
tecting presentation time stamps in the packetized ele- 
mentary streams and extracting the presentation time 
st n = there i; and 3) uitforseie 
presentation time stamps associated with a first one of 
the plurality of packetized elementary streams and 



n nsmitting II i ation lirr •? stamps to a 

clock generation circuit, wherein the clock generation 
i i t t i c I < iqn it u< c by 

a first decoder to decode the first packetized elementary 
stream. The clock generation circuit further generates a 
second reference clock signal synchronized to the first 
reference clock signal, wherein the second reference 
clock signal is used by a second decoder to decode a 
second packetized elementary stream in synchroniza- 
tion with the first packetized elementary stream. 
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Description 

CROSS-REFERENCE TO RELATED APPLICATIONS 

[0001] The present invention is related to those dis- 
closed in the following United States Patent Applica- 
tions: 

1. Serial No. [Docket No, 01-S-016], filed concur- 
rently herewith, entitled ASYSTEM FOR MULTi- 
J lf / IN - I - i r ! \ "v ' "I i 
VIDEO RECORDER AND METHOD OF OPERAT- 
ING THE SAME;® 

2. Serial No, [Docket No. 01-5-017], filed concur- 
rently herewith, entitled AAPPARATUS AND 
METHOD FOR INDEXING MPEG VIDEO DATA TO 
PERFORM SPECIAL MODE PLAYBACK IN A DIG- 
ITAL VIDEO RECORDER®; 

3. Serial No. [Docket No. 01-S-018], filed concur- 
rently herewith, entitled ADIGITAL VIDEO RE- 
CORDER USING CIRCULAR FILE MANAGE- 
MENT AND METHOD OF OPERATION;® 

4. Serial No. [Docket No. 01-S-019], filed concur- 
rently herewith, entitled ASYSTEM FOR DETECT- 
ING START CODES IN MPEG VIDEO STREAMS 
AND METHOD OF OPERATING THE SAME;@ 

[0002] The above applications are commonly as- 
signed to the assignee of the present invention. The dis- 
closures of these related patent applications are hereby 
incorporated by reference for all purposes as if fully set 
forth herein. 

TECHNICAL FIELD OF THE INVENTION 

[0003] The present invention is generally directed to 
digital video playback device and. more specifically, to 
an apparatus and method, for use in a digital video re- 
corder (DVR). for synchronizing video and audio 
streams during playback of a recorded television pro- 
gram. 

BACKGROUND OF THE INVENTION 

[0004] Digital video recorder (DVR) systems are be- 
coming increasingly popoiar with consumers. Digital 
video recorder systems use magnetic hard disk drives 
jther than ma i ; >ro 

g an or ex ecorder and the 

TiVOJ recorder record television programs in digital for- 
mats on hard disk drives using, for example, MPEG-2 
compression. Also, some DVR systems may record on 
a readable/writable digital versatile disk (DVD) rather 
than a magnetic disk. 

[0005] MPEG-2 compression is essential for storing 
a recorded television program. An uncompressed video 
program would require an enormous (and impractical) 
amount of storage space on a disk drive. 
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[0006] Sin a equire an enor- 

mous amount of bandwidth to provide services such as 
video-on-demand, videoconferencing, and the like. 
However, the inhere * hr ttionso n« 
5 are a primary inhibitor to the performance of such sys- 
tems. 

[0007] Therefore, compression and communication 
standards have been developed to overcome the band- 
width limitations of conventional communication nei- 

io 'i s> 1 < ,i , ssion of v 

eo and audio data and the delivery of control data in a 
single bit stream transmitted in a frequency band that 
would before only accommodate an analog program. 
[0008] Moving Picture Experts Group (MPEG) is a 

15 family of audio and video compression standards. \r, the 
MPEG « 

within a given picture {i.e., spatial compression) and be- 
tween pictures (i e.. I r\ N : compression). Spatial 
compression is accomplished by conversion of a digital 

?o image from the time domain to the frequency domain by 
a discrete cosine transform, quantization, variable 
length coding, and Huffman i c ling, "temporal compres- 
sion is accomplished via a process referred to as motion 
compensation in which a motion vector is used to de- 

25 scribe the translation of picture elements between pic- 
ture frames. 

[0009] ISO 13818-1 is the transport layer portion of 
the MPEG-2 standard, and specifies: i) packetization of 
audio and video elementary bit streams into packetized 

30 elementary streams (PESs), and ii) combination of au- 
dio and video PESs into a single time division or packet 
multiplexed bit stream for transmission and subsequent 
de-muitiplexsng into multiple bit streams for decompres- 
i and display. The single time division or packet mat- 

35 tiplexed bit stream is as shown from various conceptual 
perspectives in FIGURES 1 to 5. 
[0010] FIGURE 1 illustrates a conceptual block dia- 
gram (generally designated 100) of the packetization of 
audio and video elementary bit .streams and packet mat- 

40 tiplexing according to an exemplary embodiment of the 
prior art. Distinct elementary streams are termed in au- 
dio encoder 105, video encoder 110, auxiliary (or other 
■datai source 115, and systems oata source 120. Each 
of these elementary streams is packetized into a pack- 

45 eiized elementary stream (PES). The packetized ele- 
mentary streams of audio data, video data, and the 
packets of other data and systems data are packet mul- 
tiplexed by a MPEG-2 multiplexor into an MPEG-2 sys- 
tem stream. 

so [0011] FIGURE 2 illustrates a conceptual block dia- 
gram of an exemplary time-division or packet-multi- 
plexed bit stream (generally designated 200) according 
to an exemplary embodiment of the prior art. Bit si ream; 
200 comprises a transport packet stream, wherein each 

55 packet Illustratively comprises packet header 205 and 
payload 210 (i.e., packet data bytes) and, according to 
FIGURE 2, optional adaptation field 215. An MPEG-2 
bit stream comprises two layers, namely, a system layer 
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(also referred to as an ouier layer, a control layer, or the 
like) and a compression layer (also referred io as arc in- 
ner layer, a payload layer, a data layer, or the like). 
[0012] The MPEG-2 system layer facilitates (i) multi- 
plexing one or more programs made up of related audio 
and video bit streams into a single bit stream for trans- 
mission through a transmission medium, and (ii) de-mul- 
tiplexing of the single bit stream into separate audio and 
video program bit streams for decompression while 
maintaining synchronization. The system layer defines 
data stream syn > control and synchroniza- 

tion and interleaving of the video and audio bit streams. 
The system layer is capable of: i) video and audio syn- 
chronization, ii) stream multiplexing, iii) packet and 
stream identification, iv) error detection, v) buffer man- 
agement, vi) random access and program insertion, vii) 
pnvate data, viii) conditional access, and i interoper- 
ability with other networks, such as those using asyn- 
chronous transfer mode (ATM). The MPEG-2 compres- 
sion layer comprises the coded video and audio data 
streams. The system layer provides control data for mul - 
tiplexing and de-multiplexing interleaved compression 
layers and, in doing so, defines those functions neces- 
sary for combining the compressed data streams. 
[0013] FIGURE 3 illustrates a conceptual block dia- 
gram of an MPEG-2-compliant decoding system (gen- 
erally designated 300) according io an exemplary em- 
bodiment of the prior art. The components of decoding 
system 300 are well known to the skilled in the art and 
are therefore introduced for illustrative purposes only. 
Discussion of the functionality of these components will 
therefore be limited. 

[0014] Decoding system 300 receives bit stream 200 
as an input to system decoder 305. System decoder 305 
de-multiplexes the system layer data of bit stream 200 
into the compressed audio layer, the compressed video 
layer, and control data. The exemplary compressed au- 
dio layer data and video layer data are transferred to 
audio data buffer 310a and video data buffer 31 Ov, re- 
spectively. The audio layer data is subsequently proc- 
essed in audio decoder control block 315a and audio 
decoder 320a. The video layer data Is subsequently 
processed In video decoder control block 315v and vid- 
eo decoder 320v. Exemplary control data is shown as 
program clock recovery (PCR) data, enable data, and 
startup values. 

[001 5] The MPEG-2 system layer supports a plurality 
of functions, namely, i) packet multiplexing and de-mul- 
iplexing 5 iuI exe I reams synchr 

nous display of multiple coded bit streams, iii) buffer 
management and control, Iv) time recovery and identi- 
fication, v) random access, vl) program insertion, vii) 
conditional access, and viii) error tracking. 
[0016] The MPEG-2 standard specifies two types of 
layer coding, namely, a program stream (PS) layer cod- 
ing for relatively loss-less environments (e.g., 
CD-ROMS, DVDs, etc) and transport stream (TS) layer 
coding for lossy environments (e.g., cable television. 
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satellite television, or other broadcast environments). 
Referring back to FIGURE 2, bit stream 200 is illustra- 
tively a transport stream (TS) consisting of a plurality of 
TS packets divided into a packet header, an optional ad- 

5 aptation field, and the associated packet data (or pay- 
load). By contrast FIGURE 4 illustrates a conceptual 
block dlagra 1 nated 400) ac- 

cording to an exemplary smbodiment of the prior art. 
[0017] Packetized elementary stream (PES) 400 

10 comprises packet header 405, optional PES header 
410, and associated packet data 415. Packet header 
405 comprises packet start code prefix 420, stream 
Identifier (ID) 425 a' j PE « th ndlcator430 
In accord herewith, all of the fields after PES packet 

15 length indicator 430 are optional. PES header 410 in- 
cludes a presentation time stamp (PTS) field, a decod- 
ing time stamp (DTS) field, an elementary stream clock 
reference (ESCR) field, a elementary stream (ES) rate 
field, a DSM trick mode field, a copy information field, a 

?o prior PES dock recovery field, an extension field, and 
stuffing bytes. 

[0018] Packet start code prefix 420 provides packet 
synchronization. Stream ID 425 provides packet identi- 
fication and payload identification. PTS/DTS flag fields 
is 435 and PTS/DTS fields 440 provide presentation syn- 
chronization. Data trans n i , , c ■ 'ough the pack- 
et/header length 445, payload 415, and stuffing fields 
450. Scramble control field 455 facilitates payload de- 

30 [0019] FIGURE 5 illustrates a conceptual block dia- 
gram of an alternate time-division or packet-multiplexed 
bit stream (generally designated 200) according to an 
exemplary embodiment of the prior art. Bit stream 200 
comprises access units 500, PES packets 400, and a 

35 plurality of TS packets 505. Bit stream 200 illustrates a 
layering relationship among access units 500, PES 
packets 400, and TS packets 505. 
[0020] The TS layer operates io combine programs 
made up of PES-coded data with one or more independ- 

40 ent time bases into a single stream. 

[0021] In accord with MPEG-2, a specific program 
does not require a unique time base, but if it does have 
a unique time base, the time base is the same for ail of 
the elements of that program. 

is )Q22] 5 iayei layer poi > 

MPEG-2 time division or packet multiplexed stream up- 
on which the transport or program streams are logically 
constructed. The PES layer provides stream specific op- 
erations and supp - ng s common base 

so of conversion between program and transport streams, 
ii) time stamps for videc , diosy chronization and 
associated timing, especially for associated audio and 
video packets making up a broac 
iation, or program (collectively hereafter Programs), and 

55 having a commor time ba sc ) strea 

stream multiplexing and de-multipiexing, and iv) such 
services as scrambling, VCR functions, and pnvate da- 
ta. 
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[0023] FIGURE 5 further illustrates that, in accord with 
MPEG-2, each video or audio elementary stream (ES) 
is PES-packetized before insertion into a transport 
stream (TS). Elementary streams are continuous and 
PES packets containing an ES are generally of fixed s 
lengths. Typically, video PES packets are on the order 
of tens of thousands of bytes and audio PES packets 
are on the order of thousands of bytes. However, video 
PES packets can also be specified as of undefined 
length. ES d ta that s, access units 50( s irst en- 10 
capsulated into PES packets, which are, in turn, inserted 
into TS packets. 

[0024] A transport stream may contain one or more 
i ndenl, indiv 5 c < 1 ) i > c j< as ^dividual 
broadcast television programs, whereby each individual is 
program may have its own time base, and each stream 
comprises an Individual program having lis own packet 
identification (PID). Each separate individual program 

x r i ( ' i ill 

a common time base. While not illustrated in the PRIOR ?o 
ART figures, different transport streams may be com- 
bined into a single system TS. 
[0025] At the transport layer, the transport sync byte 
provides .> 3 1 e PID field data pro- 

vines pari.m elm i mem ut en] am; - :m 

quence integrity data. The PID field is operable to collect 
the packets of a stream and reconstruct the stream. 
Continuity counters and error indicators provide packet 
segue eli egrity and error detection. The payioad unit 
start indicator and adaptation control are used for pay- so 
load synchronization, while the discontinuity indicator 
and program clock reference (PGR) fields are used for 
playback synchronization. The transport scramble con- 
trol field tar 1 1 rami) ng Private data 
transfer Is accomplished through the pnvaie data Hag -3 
and private data bytes. The data bytes are used for pri- 
v ) y ! ! c ' 3 M ^'1^1,. 
used to round out a packet. 

[0026] A transport stream is a collection of transport 
strean acke 1 ^ e ese tables 

1 i 1 i - P I it 
when a TS is created ai the multiplexor. These tables 
completely define the content of the stream. Two of the 
es of ji in if soc at or table (PAT 

and the program map table (PMT). The PAT operates 45 
as a table of contents for the TS that contains a unique 
identifier for the stream, a version number enabling dy- 
namic changes of the PAT and the TS. and an associa- 
tion table of pairs of values. T he pairs of values, PN, and 
PMT-PID, are the program number (PN) and the PiD of so 
the tables containing the program, 
[0027] The PMT, on the other hand, describes all 
streams comprising a program. Each entry in the PMT 
is related to one program. The PMT provides a mapping 
between packets and programs, and contains a pro- 55 
gram number that identifies the program within the 
stream, a descriptor to carry private information about 
the program, the PID c - mi tain the syn- 



chronization information, a number of pairs of values (e. 
g., stream type (ST), Data-PID) which, for each stream, 
specify the S"" ' f 

data of that stream or program (Data-PID). 
[0028] Collectively, these tables are used to process 
a particular program. Aiany point in time, each program 
has a unique PID in the PMT, which provides the PIDs 
for the selected program=s audio, video, and control 
streams. The streams with the selected PiDs are ex- 
tracted and delivered to the appropriate buffers and de- 
coders for reconstruction and decoding. 
[0029] Achieving and maintaining clock recovery and 
synchronization Is a 1 s i with 0 an 

c • bit ;tn ii !n a< , c c 11 ■ \ PEG 2 si andard, 
an end-to-end constant delay timing mode! digital image 
and audio data take the same amount of time to pass 
through the system from encoder to decoder. The sys- 
tem layer contains timing information that requires con- 
stant delay. The dock 1 e 

erence (PGR) and the time stamps are the PTS and 
DTS. 

[0030] Synchronization is accomplished using the 
program clock reference (PGR) data field in the TS ad- 
aptation field. PGR is typically a 42-bit field that is coded 
in two parts, a PGR base having a 33-bit value in units 
of SO kHz, and a PGR extension having a 9-bit extension 
in units of 27 MHz, where 27 MHz is the system clock 
frequency. As a general rule, the first 33 bits of the first 
PGR received by the decoder Initialize the counter in a 
> 1 - 5 subs equent PGR values are com- 
pared to clock values forfine adjustment. The difference 
between the PGR and the local clock can be used to 
drive a voltage-controlled oscillator, or a similar device 
or function, for example, to speed up or slow down the 
local clock. 

[0031] Audio and video synchronization is typically 
i dr i i jh the presentation time stamp In- 
serted in the PES header. The presentation time stamp 
is a 33-bit value In units of 90 kHz, where 90 kHz is the 
27 MHZ system clock divided by 300. The presentation 
time stamp value indicates the time that the presentation 
unit should be presented to the user. 
[0032] ' ids ' , 5 o "i I.i-IG. Y[n ( 

ant digital video systems and HDTV compliant digital 
video systems), the transport layer Is used to carry audio 
data, video data, and system data streams, in packets, 
to the individual decoders and system memory. The in- 
dividual streams are multiplexed together with a set of 
buffer assumptions as defined in the MPEG-2 System 
Layer specification. 

[0033] In MPEG transport and program streams, syn- 
chronlzatior - , streams is ac- 

complished - 1 PTS) in the 

program streams and transport streams. As noted 
above, time stamps are generally in units of 90 KHz, but 
the system cloc - >CR), the program clock 

reference (PGR) and the optional elementary stream 
clock reference (ESCR) have extensions with a resolu- 
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lion of 27 MHz. Decoding of N elementary streams is 
synchro" I ! h i ' 1 it i 

common master time base f ather than by adjusting the 
decoding of one stream to match that of another. The 
master time base may be one of the clocks of the H de- 
coders, the clock of the data source, or it may be some 
external clock. 

[0034] Each program in a transport stream, which 
may contain multiple programs, may have its own time 
base. The time bases of different programs within a 
transport stream may be different. Because presenia- 
i | he decoding of indivic j! el- 
ementary streams, they reside in the PES packet layer 
of both the transport streams and the program streams 
End-to-end synchronization occurs when encoders 
save time stamps at capture time, the l:me stamps; prop- 
agate with associated coded data to decoders, and the 
decoders use those time stamps to schedule presenta- 
tions. 

[0035] As noted above, synchronization of a decoding 
system with a channel is achieved through the use of 
the system clock reference (SCR) in the program stream 
and by its analog, the program clock reference (PGR), 
in the transport stream. The SCR and PGR are time 
stamps encoding the timing of the bit stream itself, and 
are derived from the same time base used for the audio 
and video presentation time stamp values from the 
same program. Since each program may have its own 
time base, there are separate PGR fields for each pro- 
gram in a transport stream containing multiple pro- 
grams. In some cases it may be possible for programs 
to share PGR fields. A program shall have one and only 
one PGR time base associated with it. 
[0036] Unfortunately, in a digital video recorder with 
packetized elementary stream (PES) storage, the sys- 
tems time clock (STC) and the program clock reference 
(PGR) are lost when a video program is stored 1o the 
disk drive. The program is played back at a later time (i. 
e., time-deferred viewing). Therefore, there is a need in 
the art for an improved digital video recorder that imple- 
ments a new synchronization apparatus and algorithm 
for audio and video s hroni > 

SUMMARY OF THE INVENTION 

[0037] To address file above-discussed deficiencies 
of the prior art, it is a primary object of the present in- 
vention to provide time-deferred audio-video synchroni- 
zation o c I i i is. T ach e <=> 
audio-video 3> i program clock 
reference, the ) i ales one of the 
streams of the program as the master, which means that 
the program time stamp (PTS) values of the master are 
used as s^osti gra n clock reference. The 
present Invention maintains a counter representing the 
master PTS values as free-running and synchronizes 
all the other streams on the master PTS values. The pro- 
pose 3 s "lu > at e processing 



is used as with the PGR values, except that a master 
PTS is used instead of the PGR. The proposed solution 
also is robust, as the PTS gaps are taken into account. 
[0038] Accordingly, there Is disclosed an MPEG de- 

5 coder according to the principles of the present inven- 
tion. According to an advantageous embodiment of the 
present invention, the MPEG decoder comprises: 1) a 
packetized elementary stream (PES) interface capable 
of receiving a plurality of packetized elementary streams 

m r ingle vide n; 2) a presenta- 

tion time stamp (PTS) detection circuit capable of de- 
tecting presentation time stamps in the packetized ele- 
nentary streams and e esentatio time 

stamps therefrom; and 3) a selection circuit capable of 

15 selecting presentation time stamps associated with a 
first one of the plurality of pat ket zed eiem t 
streams and transmitting the selected presentation time 
stamps to a clock generation circuit, wherein the clock 
generation circuit generates a first reference clock sig- 

?o nal used by a first decoder to decode the first packetized 
elementary stream. 

[0039] According to one embodiment of the present 
invention, the > i> t < 1 thor generates 

a second reference clock signal synchronized to the first 
?s reference clock signal and wherein the .second refer- 
ence clock signal Is used by a second decoder to de- 
code a second packetized elementary stream in syn- 
chronization with the first packetized elementary 

30 [0040] According So another embodiment of the 
present invention, the selected presentation time 
stamps are video presentation times stamps and the 
first decoder is a video decoder. 
[0041] According to still another embodiment of the 

35 present invention, the second decoder is an audio de- 
coder, 

[0042] According to yet another embodiment of the 
present invention, the selected presentation time 
stamps are audio presentation times stamps and the 
40 first decoder is an audio decoder. 

[0043] According to a further embodiment of the 
present invention, the second decoder is a video decod- 
er. 

[0044] According to a still further embodiment of the 
is t n fk 1 Mi ui ]t n« r< k 

the second reference clock signal by synchronizing 
c eseniatlc i t i e sta i vi the second 

packetized elementary stream with the selected pres- 
entation time stamps associated with the first pack- 
so etized elementary stream, 

[0045] The foregoing has outlined rather broadly the 
features and" ' i iges of the present inven- 

tion so that those skilled in the art may better understand 
the detailed description of the invention that follows. Ad- 
55 ditionai features and advantages of the invention will be 
described hereinafter that form the subject of trie claims 
of the invention. Those i k 'loc r> t 
ate that they may readily use the conception and the 
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specific embodiment disclosed as a basis for modifying 
or designing other structures for carrying out the same 
purposes of the present invention. Those skilled in the 
art should also realize that such equivalent construc- 
tions (io not depart from the spirit and scope of the in- 5 
vention in its broadest form. 

[0046] Before undertaking the DETAILED DESCRIP- 
TION OF THE INVENTION below, it may be advanta- 
geous io set forth definitions of certain words and phras- 
es used throughout this patent document, the terms "m- io 
t i if ind M st as well i is aiiv? iereo 
mean "inclusion without limitation"; the term "or", is In- 
clusive, meaning "and/or"; the phrases "associated 
with" and "associated therewith", as well as derivatives 
thereof, may mean "include", Abe included within®, "in- is 
tereonneci with*, "contain", Abe contained within®, 
"connect to or with", "couple to or with", Abe communi- 
cable with®, "cooperate with", "interleave", "juxtapose", 
"be prox:rnate to", "be bound to or with", "have", "have 
a property of, or the like; and the term "controller" in- ?o 
eludes any device, system or par; thereof that controls 
at least one operation, such a device may be Implement- 
ed in hardware, firmware or software, or some combi- 
nation of at least two of the same. In particular, a con- 
troller may comprise a data processor and an associat- ?s 
(. 3 l i r t 1 ' £ > t i ^ 

ed by the data processor. It should be noted thai the 
functionality associated with any particular controller 
may be centralized or distributed, whether locally or re- 
motely. Definitions tor certain words and phrases are 30 
provided throughout this patent document, those of or- 
dinary skill in the art should understand thai in many, if 
not most instances, such definitions apply to prior, as 
well as future uses of such defined words and phrases. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0047] For a more complete understanding of the 
present invention, and the advantages thereof, refer- 
ence is now made to the following descriptions taken in 40 
conjunction with the accompanying drawings, wherein 
like numbers designate like objects, and in which; 



an alternate time-division or packet-multiplexed bit 
stream according to an exemplary embodiment of 
the prior art; 

FIGURE 8 illustrates an exemplary digital video re- 
corder (DVR) system and an associated television 
set according to one embodiment of the present in- 
vention; 

FIGURE 7 illustrates the exemplary digital video re- 
corder (DVR) system in greater detail according io 
one embodiment of the present invention; 
FIGURE 8 illustrates an MPEG-2 decoding system 
in the exemplary DVR system according to an ex- 
emplary embodiment of the present invention; and 
FIGURE 9 is a flow chart illustrating the operation 
of the MPEG-2 decoding system according to an 
exemplar* il n pi invention. 

DETAILED DESCRIPTION OF THE INVENTION 

[0048] FIGURES 1 through 9, discussed below, and 

i i ts used to des ie the prin 
pies of the present invention in this patent document are 
by way of illustration only and should not be construed 
in any way so as io limit the scope of the invention. 
Those skilled in the art will understand that the principles 
of the present Invention may be implemented in any suit- 
ably arranged video playback device. 
[0049] FIGURE 6 Illustrates exemplary digital video 
recorder (DVR) system 650 and television set 605 ac- 
cording to one embodiment of the present invention. 
Digital video recorder system 650 receives incoming tel- 
evision signals from an external source, such as a set- 
top box of a cable television service provider (Cable Co.) 
or the phone company, a local antenna, the Internet, a 
satellite television service provider, or an attached DVD 
or VHS iape player, and transmits a viewer-selected 
channel to television set 605. In RECORD mode, digital 
video recorder system 650 may demodulate an incom- 
ing radio frequency (RF) television signal to produce a 
baseband video Signal thai is recorded and stored on a 
storage medium within or connected to video playback 
device 650. in PLAY mode, digital video recorder sys- 
tem 650 retrieves a stored baseband video signal (i.e., 
xr it - - , i . n torage medium 

and transmits the retrieved program to television set 
605. 

[0050] According to an exemplary embodiment of the 
present Invention, digital video recorder (DVR) system 
650 is a disk drive-based device, such as a Replay TVJ 
recorder or a TiVOJ recorder, DVR system 650 stores 
and retrieves the incoming television signals to and from 
a computer iagn« n< ather than a magnetic 

cassette tape, in st o s 

back device 650 may store and retrieve from a local 
read/write (R/W) digital versatile disk (DVD) or R/W 
CD-ROM Thus, the local storage medium may be fixed 
(e.g., hard disk drive) or removable (e.g., DVD, 
CD-ROM). 



FIGURE 1 Illustrates a conceptual block diagram of 
the packeiizaiion of audio and video elementary bit 45 
streams and packet multiplexing according to an 
exemplary embodiment of the prior art; 
F SURE st S3 concept )lot lia | < 
an exemplar-, time-division or packer-multiplexed 
bit stream according to an exemplary embodiment so 
of the prior art; 

FIGURE 3 Illustrates a conceptual block diagram of 
an MPEG-2-c 1 - en according 

to an exemplary embodiment of the prior art; 
FIGURE 4 illustrates a conceptual block diagram of 55 
a PES according to an exemplary embodiment of 
the prior art; 
FIGURE i i 
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[0051] Digital video recorder system 650 comprises 
infrared (!R) sensor 860 that receives commands {such 
as Channel Up, Channel Down, Volume Up, Volume 
Down, Record, Play, Fast Forward (FF), Reverse, and 
' p.) from a remote control dev > I s 

viewer. Television set 605 is a conventional television 
comprising screen 610, infrared (IR) sensor 615, and 
one or more man- ated by a dotted 

line) IF: sensor 615 also receives commands (such as 
volume up, volume down, power ON/OFF) from a re- ~>o 
mote control device operated by the viewer. 
[0052] It should be noted that DVR system 650 Is not 
limited to receiving a particular type of Incoming televi- 
sion signal from a particular type of source. As noted 
above, the externa! source may be a cable service pro- 15 
vider, a conventional RF broadcast antenna, a satellite 
dish, an Internet connection, or another local storage 
device, such as a DVD player. Thus, the incoming signal 
may be a conventional digital signal, such as MPEG-2. 
an conventional NTSC analog signal, or digital internet 20 
protocol (IP) packets of any other communication pro- 
tocol. However, for the purposes of simplicity and clarity 
in explaining the principles of the present invention, the 
- fc ow shall generally be directed to an 
embodiment in which DVR system 650 receives incorn- is 
ing television signals (analog 3nd/or digital) from a cable 
service provider. None' m t I ^ i 

will understand that the principles of the present inven- 
tion may readily be adapted for use with wireless broad- 

isio local storage sy stems, an incom- 30 

- <ti- if <=> n ' d_ 1 i 

like. 

[0053] FIGURE 7 illustrates exemplary digital video 
recorder (DVR) system 650 in greater detail according 
to one embodiment of the present invention. DVR sys- 35 
tern 650 comprises IR sensor 660, video processor 7 1 0, 
optional MPEG-2 encoder 720, hard disk drive 730, 
MPEG-2 decoder/NTSC encoder 740. DVR system 650 
further comprises vicieo buffer 760 and video recorder 
(VR) controller 750. 40 
[0054] VR controller 750 directs the overall operation 
of DVR system 650, including View mode, Record 
mode, Play mode, Fast Forward (FF) mode, Reverse 
mode, Pause mode, and Skip mode, among others. !n 
View mode, VR controller 750 causes the incoming lei- « 
evision signal from the cable service provider 1o be de- 
modulated and processed by video processor 710 and 
transmitted to television set 605, without storing or re- 
trieving from hard disk drive 730. 
[0055] 1 nt circuit iVRs n 650 varies 
i er the ex ernal bi >ad -si ^ c a e 

ceived by DVR system 650 is digital, such as MPEG-2, 
or analog, such as NTSC. For example, if the external 
broadcast s ivibei vice provid- 

> ' " v .(nil ( i) )cpav r 710 may be, for 55 
p m ' ' v sor which 

contains radio frequency (RF) front-end circuitry for re- 
ceiving incoming analog television signals, tuning to a 



user-selected channel, and converting the selected RF 
signal to a baseband television signal (e.g., super video 
signal) suitable for display on television set 605. Video 
processor 710 also Is capable of receiving a convention- 
al NTSC, PAL. orSECAM signal from MPEG2 decoder/ 
NTSC encoder 740 (after buffering in video buffer 760) 
during Play mode and transmitting baseband television 
signal (e.g., super video signal) to television set 605. 
Alternatively, if the external broadcast signal is a digital 
signal, such as an MPEG-2 signal, video processor 710 
may be a conventional digital TV decoder, such as an 
MPEG-2 decoder that converts the digital input signal 
j a baseoano ifle i vdisp bievisio 

set 605. 

[0056] In Record mode, VR controller 750 causes the 
incoming teli , n 3rd disk drive 

730. MPEG-2 encoder 720 is optional provided if the ex- 
ternal broadcast signal Is an analog signal. For example, 
under the control of VR controller 750, MPEG-2 encoder 
720 may receive an incoming NTSC-type analog televi- 
sion signal and convert the received broadcast signal to 
MPEG-2 format for storage on hard disk drive 730. Al- 
ternatively, If the external broadcast signal is already a 
digital signal, such as MPEG-2, then the broadcast sig- 
nal may be stored directly to hard disk drive 730 and 
MPEG-2 may be omitted, in Play mode, VR controller 
750 directs hard disk drive 730 to stream the stored tel- 
evision signal (i.e., program) to MPEG-2 decoder/NTSC 
encoder 740, which converts the MPEG-2 data from 
hard disk drive 730 to, for example, a super video (S-Vid- 
eo) signal that is buffered in video buffer 760 before vid- 
eo processor 710 transmits it to television set 605, 
[0057] !t should be noted that the choice of the MPEG- 
2 standard for MPEG-2 encoder 720 and MPEG-2 de- 
coder/NTSC encoder 740 Is by way of Illustration only. 
In alternate embodiments of the present invention, the 
MPEG encoder and decoder may comply with one or 
more of the MPEG- i, MPEG-2, MPEG-4, and MPEG-7 
I 1 tar i i it i i ia;e embodin I he die 
iial television encoding and decoding circuitry may com- 
ply with some other digital television standard. 
[0058] For the purposes of this application and the 
claims that follow, hard disk drive 730 is defined to in- 
clude any mass storage device thai is bold readable and 
writable, including conventional magnetic disk drives, 
and optical disk drives tor react/write digital versatile 
disks (DVD-RVV), re-writable CD-ROMs, and the like. In 
fact, hard disk drive 730 need not be fixed in the con- 
ventional sense thai is permanently embedded in DVR 
system 650. Rather, hard disk drive 730 includes any 
mass storage device that is dedicated to DVR system 
650 for the purpose of storing recorded television pro- 
grams. Thus, hard disk drive 730 may include an at- 
tached peripheral drive or removable disk drives (wheth- 
er embedded or attached), such as a juke box device 
that holds read/write DVDs or re-writable CD-ROMs. 
Furthermore, in an advan jgeou en xilment of the 
present invention, hard disk drive 730 may include ex- 
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ternal mas? stc - yf em 650 may 

access and control via a network connection {e.g., in- 
ternet protocol (iP) connection), including, for example, 
a disk drive in a home personal computer (PC) or a disk 
drive on 3 server a: an Internet service provider (ISP). 5 
[0059] As noted above, synchronization of a decoding 
system with a channel is achieved through the use of 
the system do ?) thep istrearr 

and by its analog, the program clock reference (PGR), 
iri the transport stream. However, as discussed in the 10 

1 und 3 3ve in a ;k tal \ ie 3 rec< der ([ v'F 
system with packetized elementary stream (PES) stor- 
age, the systems time clock (STC) and the program 
clock reference (PGR) ere lost when a recorded televi- 
sion program is stored to the disk drive. Accordingly, the 15 
present invention provides an improved video and audio 
synchronization circuit and related method for use in 
MPEG-2 deccder/NTSC encoder 740 in DVR system 
650. 

[0060] FIGURE 8 illustrates MPEG-2 decoding sys- 20 
tern 800 in MPEG-2 decoder/NTSC encoder 740 ac- 
cording to an exemplary embodiment of the present in- 
vention. Most of the components of MPEG-2 decoding 
system 800 are well known to those skilled in the art. 
During playback, MPEG-2 decoding system 800 re- 25 
ceives packetized elementary stream (PES) data from 
hard disk drive 730 as an input to system decoder and 
extract PTS block 805. System decoder and extract PTS 
block 805 de-muitiplexes PES data into the compressed 
audio layer, the compressed video layer, control data, 30 
and presentation time stamps. The exemplary com- 
pressed audio layer data and video layer data are trans- 
ferred to audio data buffer 810a and video data buffer 
81 Ov, respectively. The audio layer data is subsequently 
processed in audio decoder control block 81 5a and au- 35 
dio decoder 820a. The video layer data Is subsequently 
processed in video decoder control block 8l5vand vid- 
eo decoder 820v. 

[0061] Exemplary control data is shown as video 
• re eolation time stamp (PTSv) data, audio presents- 40 
tion time stamp (PTSa) data, data presentation time 
stamp (PTSd) data, enable data, and startup values. 
The PTSv data, PTSa data, and PTSd data extracted 
by system decoder and extract PTS block 805 are trans- 
ferred to the input channels of multiplexer (MUX) 880. 45 
P < ' ^ itlo lies rip (PTS )i' i ;ele< in > 
selects one of the input channels of MUX 880 to be out- 
put as a t ' 1 renee< (PGR) 
to systems time clock generator 830. 
[0062] FIGURE 9 depicts flow chart 900, which illus- -« 
trat< = the operation of MPEG-2 decoding system 800 
according to an • nbodimenf of the present 
invention. Dun 'EG-2 decoding system 
SO; recs ves ; jrahi> of PES streams associated with 
a vicieo program (process step 905) PTS master select 55 
890 designates one of plurality of PES streams as a 
master PES stream for video program (process step 
910). Systems i uses presenta- 



tion time stamp ; s substitute to 

PGR (process step 915). Presentation time stamps of 
non-master PES streams are synchronized to PTS val- 
ues of master PES stream (process step 920). The 
present invention maintains a counter representing the 
master PTS values as free-running and synchronizes 
all the other streams on the master PTS values. 
[0063] Although the present invention has been de- 
scribed in detail, those skilled in the art should under- 
stand that they can make various changes, substitutions 
and alterations herein without departing from the spirit 
5f! ,-j s ,f t , , , r , j n broadest form 



Claims 

1 . An MPEG decoder comprising: 

a packetized elementary stream (PES) inter- 
face capable of receiving a plurality of pack- 
etized elementary streams associated with a 
single video program; 

a presentation time stamp (PTS) detection cir- 
cuit capable of detecting presentation time 
stamps in said packetized elementary streams 
and extracting said presentation time stamps 
therefrom; and 

a selection circuit capable of selecting presen- 
tation time stamps associated with a first one 
of said plurality of packetized elementary 
streams and transmitting said selected presen- 
tation time stamps to a clock generation circuit, 
wherein said clock generation circuit generates 
a first reference clock signal used by a first de- 
coder to decode said first packetized elemen- 
tary stream, 

2, A digital video recorder capable of playing back a 
recorded television program stored as packetized 
elementary streams, said digital video recorder 
comprising: 

a video processor capable of receiving an in- 
coming television program and converting said 
Incoming television program to a baseband vid- 
eo signal capable of being displayed on a tele- 
vision set coupled to said digital video recorder; 
a storage disk capable of storing said incoming 
television program as packetized elementary 
streams during recording; and 
an MPEG < 

ketized 1 e: - , 

ierface capable of receiving a plurality of 
packetized elementary streams associat- 
ed with said recorded television program 
from said storage disk during playback; 
a presentation time stamp (PTS) detection 
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c re Jit capable oi te j 
time stamps in said packetizeci elementary 
streams and extracting said presentation 
time stamps therefrom; and 
a selection circuit capable of selecting 
presentation time stamps associated with 
a first one of said plurality of packetized e! - 
ementary streams and transmitting said 
selected presentation time stamps to 3 
/k 1 > 1 c % 

generation circuit generates a first refer- 
ence clock signal used by a first decoder 
to decode said first packetized elementary 
stream. 

The decode' or digits dec recorder as set forth in 
Claim 1 or 2 wherein said clock generation circuit 
further generates a second reference clock signal 
synchronized to said first reference clock signal and 
wherein said second reference i eianai 
by a second decoder to decode a second pack- 
etized elementary stream in synchronization with 
said f t i I i ^ 1 1 , , ■>! 

The decoder or digital video recorder as set forth in 
Claim 3 said clock generators circuit generates said 
second reference clock signal by synchronizing 
presentation time stamps associated with said sec- 
ond packetized elementary stream with said select- 
ed presentation time stamps associated with said 
first packetized elementary stream. 



7. The method or recorder or decoder as set forth In 
Claim 3 or 5 wherein the selected presentation time 
stamps are video presentation times stamps and 
the first decoder is a video decoder. 

8. The method or recorder or decoder as set forth in 
Claim 7 wherein the second decoder is an audio de- 
coder. 

9. The method or recorder or decoder as set forth in 
Claim 3 or 6 wherein the selected presentation time 
stamps are audio presentation times stamps and 
the first decoder is an audio decoder. 

10. The method or recorder or decoder as set forth in 
Claim 9 wherein the second decoder is a video de- 
coder. 

1 1 . The method as set forth in Claim 6 wherein the step 
of generating a second reference clock signal com- 
prises the sub-step of synchronizing presentation 
t ncsot i|. ? e second packetized 
elementary stream with the selected presentation 
time stamps associated with the first packetized el- 
ementary stream. 



For use!! i a i I lod for de- 

coding a television program stored in MPEG format: 

receiving in an MPEG decoder a plurality of 
packetized elementary streams associated 
with the stored television program; 
detecting presentation tune stamps in the pack- 
etized elementary streams; 
extracting the presentation time stamps from 
the packetized elementary streams; 
selecting presentation time stamps associated 
with a first one of the plurality of packetized el- 
ementary streams; and 

1 - t i t t 1 - ;enl > t i 

stamps a first reference clock signal used by a 
first decoder to decode the first packetized el- 
ementary stream. 



The melt x t 1 

ing the step i lerating a s econd reference clock 
signal synchronized to the first reference clock sig- 
nal, wherein - ference clock signal is 
used by a second decoder to decode a second 
packetized elementary stream in synchronization 
with the first packetized elementary stream. 



EP 1 289 306 A2 




10 



EP 1 289 306 A2 




EP 1 289 306 A2 




12 



EP 1 289 306 A2 



FiOOo 



FIG. 5 ? oa 

(PRIOR ART) * 



\ 500c 



40.0O \\ 205 40 P b \X 20S 4U ^ 



PACKDfi 



0 J 1 



210 ; 



\ \ \ 210 



505o 505b 505c 505d _ 5QSe 505? S05g 




620 



13 



EP 1 289 306 A2 



FIG. 7 



mJi 



i 730 

JLZ 



740 


NTSC f. 

ZT 



BUFFER 



TO IV ( 



905 J 1 

»{ 



»kg finback, mpfo-2 pfcocffg 



PPF iaASTO? SL-IICT 330 FFFiGFAFFP: AM; 

OF \ ^ ^ H w A 

MASTER PtS STREAM FOR ViQEO PROGRAM 



PRFSFNTA 


m mi sr 


>M ' ' f F 




PES SIR 
VAL 


tAMS ARf: S> 
JFS 0? MASS 


FG-JF ; 
" RFS ST 


5 10 PTS 
! ?EAM 



EP 1 289 306 A2 




15 



